Evaluating the Performance of a Commercially Available AI Algorithm for Automated Detection of Pulmonary Embolism on CECT and CTPA of COVID-19 Patients

Objective To investigate the performance of a commercially available artificial intelligence (AI) algorithm for detection of pulmonary embolism (PE) on contrast-enhanced CTs in patients hospitalized for COVID-19. Patients & Methods Retrospective analysis was performed of all contrast-enhanced chest CTs on patients admitted for COVID-19 between March 2020 and December 2021. Based on the original radiology reports, all PE-positive exams were included (n=527). Using a reversed flow single gate diagnostic accuracy case-control model, a randomly selected cohort of PE-negative exams (n=977) was included. Pulmonary parenchymal disease severity was assessed for all included studies using a semi-quantitative system, the Total Severity Score (TSS). All included CTs were sent for interpretation by the commercially available AI algorithm, Aidoc. Discrepancies between AI and original radiology reports were resolved by three blinded radiologists, who rendered a final determination of indeterminate, positive, or negative. Results A total of 78 studies were found to be discrepant, of which 13 (16.6%) were deemed indeterminate by readers and excluded. The sensitivity and specificity of AI was 93.2%;(95% confidence interval [CI] 90.6-95.2%), and 99.6%;(95% CI 98.9-99.9%), respectively. AI's accuracy for all TSS groups (mild, moderate, severe) was high (98.4%, 96.7%, and 97.2%, respectively). AI was more accurate in PE detection on CTPAs vs CECTs (P < .001), with optimal HU of 362 (P=.048). Conclusion The AI algorithm demonstrated high sensitivity, specificity, and accuracy for PE on contrast enhanced CTs in COVID-19 patients regardless of parenchymal disease. Accuracy was significantly affected by the mean attenuation of the pulmonary vasculature. How this affects the legitimacy of the binary outcomes reported by AI is not yet known.

Evaluating the Performance of a Commercially Available Artificial Intelligence Algorithm for Automated Detection of Pulmonary Embolism on Contrast-Enhanced Computed Tomography and Computed Tomography Pulmonary Angiography in Patients With Coronavirus Disease 2019.

Objective: To investigate the performance of a commercially available artificial intelligence (AI) algorithm for the detection of pulmonary embolism (PE) on contrast-enhanced computed tomography (CT) scans in patients hospitalized for coronavirus disease 2019 (COVID-19). Patients and Methods: Retrospective analysis was performed of all contrast-enhanced chest CT scans of patients admitted for COVID-19 between March 1, 2020 and December 31, 2021. Based on the original radiology reports, all PE-positive examinations were included (n=527). Using a reversed-flow single-gate diagnostic accuracy case-control model, a randomly selected cohort of PE-negative examinations (n=977) was included. Pulmonary parenchymal disease severity was assessed for all the included studies using a semiquantitative system, the total severity score. All included CT scans were sent for interpretation by the commercially available AI algorithm, Aidoc. Discrepancies between AI and original radiology reports were resolved by 3 blinded radiologists, who rendered a final determination of indeterminate, positive, or negative. Results: A total of 78 studies were found to be discrepant, of which 13 (16.6%) were deemed indeterminate by readers and were excluded. The sensitivity and specificity of AI were 93.2% (95% CI, 90.6%-95.2%) and 99.6% (95% CI, 98.9%-99.9%), respectively. The accuracy of AI for all total severity score groups (mild, moderate, and severe) was high (98.4%, 96.7%, and 97.2%, respectively). Artificial intelligence was more accurate in PE detection on CT pulmonary angiography scans than on contrast-enhanced CT scans (P<.001), with an optimal Hounsfield unit of 362 (P=.048). Conclusion: The AI algorithm demonstrated high sensitivity, specificity, and accuracy for PE on contrast-enhanced CT scans in patients with COVID-19 regardless of parenchymal disease. Accuracy was significantly affected by the mean attenuation of the pulmonary vasculature. How this affects the legitimacy of the binary outcomes reported by AI is not yet known.

Trends in coronary calcium score and coronary CT angiography imaging volume during the COVID-19 pandemic.

OBJECTIVES: The COVID-19 pandemic disrupted the delivery of preventative care and management of acute diseases. This study assesses the effect of the COVID-19 pandemic on coronary calcium score and coronary CT angiography imaging volume. MATERIALS AND METHODS: A single institution retrospective review of consecutive patients presenting for coronary calcium score or coronary CT angiography examinations between January 1, 2020 to January 4, 2022 was performed. The weekly volume of calcium score and coronary CT angiogram exams were compared. RESULTS: In total, 1,817 coronary calcium score CT and 5,895 coronary CT angiogram examinations were performed. The average weekly volume of coronary CTA and coronary calcium score CT exams decreased by up to 83% and 100%, respectively, during the COVID-19 peak period compared to baseline (P < 0.0001). The post-COVID recovery through 2020 saw weekly coronary CTA volumes rebound to 86% of baseline (P = 0.024), while coronary calcium score CT volumes remained muted at only a 53% recovery (P < 0.001). In 2021, coronary CTA imaging eclipsed pre-COVID rates (P = 0.012), however coronary calcium score CT volume only reached 67% of baseline (P < 0.001). CONCLUSIONS: A significant decrease in both coronary CTA and coronary calcium score CT volume occurred during the peak-COVID-19 period. In 2020 and 2021, coronary CTA imaging eventually superseded baseline rates, while coronary calcium score CT volumes only reached two thirds of baseline. These findings highlight the importance of resumption of screening exams and should prompt clinicians to be aware of potential undertreatment of patients with coronary artery disease.

True-Positive Rate of RSNA Typical Chest CT Findings for COVID-19 Pneumonia in an Increasingly Vaccinated Population.

Background RSNA COVID-19 chest CT consensus guidelines are widely used, but their true positive rate for COVID-19 pneumonia has not been assessed among vaccinated patients. Purpose To assess true positive rate of RSNA typical chest CT findings of COVID-19 among fully vaccinated subjects with PCR-confirmed COVID-19 infection compared with unvaccinated subjects. Materials and Methods Patients with COVID Typical chest CT findings and one positive or two negative PCR tests for COVID-19 within 7 days of their chest CT between January 2021 - January 2022 at a quaternary academic medical center were included. True positives were defined as chest CTs interpreted as COVID Typical and PCR-confirmed COVID-19 infection within 7 days. Logistic regression models were constructed to quantify the association between PCR results and vaccination status, vaccination status and COVID-19 variants, and vaccination status and months. Results 652 subjects (median age 59, [IQR, 48-72]); 371 [57%] men) with CT scans classified as COVID Typical were included. 483 (74%) were unvaccinated and 169 (26%) were fully vaccinated. The overall true positive rate of COVID Typical CTs was lower among vaccinated versus unvaccinated (70/169 [41%; 95% CI: 34, 49%] vs 352/483 [73%; 69, 77%]; OR (95% CI): 3.8 (2.6, 5.5); P < .001). Unvaccinated subjects were more likely to have true positive CTs compared with fully vaccinated subjects during the peaks of COVID-19 variants Alpha (OR, 16 [95% CI: 6.1, 42]; P < .001) and Delta (OR, 8.3 [95% CI: 4.2, 16]; P < .001), but no statistical differences were found during the peak of Omicron variant (OR, 1.7 [95% CI: 0.27, 11]; P = .56) Conclusion Fully vaccinated subjects with confirmed COVID-19 breakthrough infections had lower true positive rates of COVID Typical chest CT findings.

Long-Term Lung Abnormalities Associated with COVID-19 Pneumonia.

In the third year of the SARS-CoV-2 pandemic, much has been learned about the long- term effect of COVID-19 pneumonia on the lungs. Approximately one-third of patients with moderate-to-severe pneumonia, especially those requiring intensive care therapy or mechanical ventilation, have residual abnormalities on chest CT one year after presentation. Abnormalities range from parenchymal bands to bronchial dilation to frank fibrosis. Less is known about the long-term pulmonary vascular sequelae, but there appears to be a persistent, increased risk of venothromboembolic events in a small cohort of patients. Finally, the associated histologic abnormalities resulting from SARS- CoV-2 infection are similar to those of patients with other causes of acute lung injury.

Chronic Pulmonary Manifestations of COVID-19 Infection: Imaging Evaluation.

This case presents a patient with severe COVID-19 pneumonia requiring intensive care unit admission and a prolonged hospital stay. The infection resulted in long-term morbidity, functional decline, and abnormal chest CT findings. The mechanisms for long-term lung injury after COVID-19 infection, imaging appearances, and the role of imaging in follow-up are discussed.

Abdominal Imaging Findings on Computed Tomography as a Tool for COVID-19 Mortality Risk Assessment: Comparison With Chest Radiograph Severity Scores.

OBJECTIVE: To quantify the association between computed tomography abdomen and pelvis with contrast (CTAP) findings and chest radiograph (CXR) severity score, and the incremental effect of incorporating CTAP findings into predictive models of COVID-19 mortality. METHODS: This retrospective study was performed at a large quaternary care medical center. All adult patients who presented to our institution between March and June 2020 with the diagnosis of COVID-19 and had a CXR up to 48 hours before a CTAP were included. Primary outcomes were the severity of lung disease before CTAP and mortality within 14 and 30 days. Logistic regression models were constructed to quantify the association between CXR score and CTAP findings. Penalized logistic regression models and random forests were constructed to identify key predictors (demographics, CTAP findings, and CXR score) of mortality. The discriminatory performance of these models, with and without CTAP findings, was summarized using area under the characteristic (AUC) curves. RESULTS: One hundred ninety-five patients (median age, 63 years; 119 men) were included. The odds of having CTAP findings was 3.89 times greater when a CXR score was classified as severe compared with mild (P = 0.002). When CTAP findings were included in the feature set, the AUCs for 14-day mortality were 0.67 (penalized logistic regression) and 0.71 (random forests). Similar values for 30-day mortality were 0.76 and 0.75. When CTAP findings were omitted, all AUC values were attenuated. CONCLUSIONS: The CTAP findings were associated with more severe CXR score and may serve as predictors of COVID-19 mortality.

Vasculopathy and Increased Vascular Congestion in Fatal COVID-19 and Acute Respiratory Distress Syndrome.

Rationale: The leading cause of death in coronavirus disease 2019 (COVID-19) is severe pneumonia, with many patients developing acute respiratory distress syndrome (ARDS) and diffuse alveolar damage (DAD). Whether DAD in fatal COVID-19 is distinct from other causes of DAD remains unknown. Objective: To compare lung parenchymal and vascular alterations between patients with fatal COVID-19 pneumonia and other DAD-causing etiologies using a multidimensional approach. Methods: This autopsy cohort consisted of consecutive patients with COVID-19 pneumonia (n = 20) and with respiratory failure and histologic DAD (n = 21; non-COVID-19 viral and nonviral etiologies). Premortem chest computed tomography (CT) scans were evaluated for vascular changes. Postmortem lung tissues were compared using histopathological and computational analyses. Machine-learning-derived morphometric analysis of the microvasculature was performed, with a random forest classifier quantifying vascular congestion (CVasc) in different microscopic compartments. Respiratory mechanics and gas-exchange parameters were evaluated longitudinally in patients with ARDS. Measurements and Main Results: In premortem CT, patients with COVID-19 showed more dilated vasculature when all lung segments were evaluated (P = 0.001) compared with controls with DAD. Histopathology revealed vasculopathic changes, including hemangiomatosis-like changes (P = 0.043), thromboemboli (P = 0.0038), pulmonary infarcts (P = 0.047), and perivascular inflammation (P < 0.001). Generalized estimating equations revealed significant regional differences in the lung microarchitecture among all DAD-causing entities. COVID-19 showed a larger overall CVasc range (P = 0.002). Alveolar-septal congestion was associated with a significantly shorter time to death from symptom onset (P = 0.03), length of hospital stay (P = 0.02), and increased ventilatory ratio [an estimate for pulmonary dead space fraction (Vd); p = 0.043] in all cases of ARDS. Conclusions: Severe COVID-19 pneumonia is characterized by significant vasculopathy and aberrant alveolar-septal congestion. Our findings also highlight the role that vascular alterations may play in Vd and clinical outcomes in ARDS in general.

Intubation and mortality prediction in hospitalized COVID-19 patients using a combination of convolutional neural network-based scoring of chest radiographs and clinical data.

Objective: To predict short-term outcomes in hospitalized COVID-19 patients using a model incorporating clinical variables with automated convolutional neural network (CNN) chest radiograph analysis. Methods: A retrospective single center study was performed on patients consecutively admitted with COVID-19 between March 14 and April 21 2020. Demographic, clinical and laboratory data were collected, and automated CNN scoring of the admission chest radiograph was performed. The two outcomes of disease progression were intubation or death within 7 days and death within 14 days following admission. Multiple imputation was performed for missing predictor variables and, for each imputed data set, a penalized logistic regression model was constructed to identify predictors and their functional relationship to each outcome. Cross-validated area under the characteristic (AUC) curves were estimated to quantify the discriminative ability of each model. Results: 801 patients (median age 59; interquartile range 46-73 years, 469 men) were evaluated. 36 patients were deceased and 207 were intubated at 7 days and 65 were deceased at 14 days. Cross-validated AUC values for predictive models were 0.82 (95% CI, 0.79-0.86) for death or intubation within 7 days and 0.82 (0.78-0.87) for death within 14 days. Automated CNN chest radiograph score was an important variable in predicting both outcomes. Conclusion: Automated CNN chest radiograph analysis, in combination with clinical variables, predicts short-term intubation and death in patients hospitalized for COVID-19 infection. Chest radiograph scoring of more severe disease was associated with a greater probability of adverse short-term outcome. Advances in knowledge: Model-based predictions of intubation and death in COVID-19 can be performed with high discriminative performance using admission clinical data and convolutional neural network-based scoring of chest radiograph severity.

Multi-population generalizability of a deep learning-based chest radiograph severity score for COVID-19.

To tune and test the generalizability of a deep learning-based model for assessment of COVID-19 lung disease severity on chest radiographs (CXRs) from different patient populations. A published convolutional Siamese neural network-based model previously trained on hospitalized patients with COVID-19 was tuned using 250 outpatient CXRs. This model produces a quantitative measure of COVID-19 lung disease severity (pulmonary x-ray severity (PXS) score). The model was evaluated on CXRs from 4 test sets, including 3 from the United States (patients hospitalized at an academic medical center (N = 154), patients hospitalized at a community hospital (N = 113), and outpatients (N = 108)) and 1 from Brazil (patients at an academic medical center emergency department (N = 303)). Radiologists from both countries independently assigned reference standard CXR severity scores, which were correlated with the PXS scores as a measure of model performance (Pearson R). The Uniform Manifold Approximation and Projection (UMAP) technique was used to visualize the neural network results. Tuning the deep learning model with outpatient data showed high model performance in 2 United States hospitalized patient datasets (R = 0.88 and R = 0.90, compared to baseline R = 0.86). Model performance was similar, though slightly lower, when tested on the United States outpatient and Brazil emergency department datasets (R = 0.86 and R = 0.85, respectively). UMAP showed that the model learned disease severity information that generalized across test sets. A deep learning model that extracts a COVID-19 severity score on CXRs showed generalizable performance across multiple populations from 2 continents, including outpatients and hospitalized patients.

United States lung cancer screening program websites: radiology representation, multimedia and multilingual content.

PURPOSE: To assess radiology representation, multimedia content, and multilingual content of United States lung cancer screening (LCS) program websites. MATERIALS AND METHODS: We identified the websites of US LCS programs with the Google internet search engine using the search terms lung cancer screening, low-dose CT screening, and lung screening. We used a standardized checklist to assess and collect specific content, including information regarding LCS staff composition and references to radiologists and radiology. We also tabulated types and frequencies of included multimedia and multilingual content and patient narratives. RESULTS: We analyzed 257 unique websites. Of these, only 48% (124 of 257) referred to radiologists or radiology in text, images, or videos. Radiologists were featured in images or videos on only 14% (36 of 257) of websites. Radiologists were most frequently acknowledged for their roles in reading or interpreting imaging studies (35% [90 of 574]). Regarding multimedia content, only 36% (92 of 257) of websites had 1 image, 27% (70 of 257) included 2 or more images, and 26% (68 of 257) of websites included one or more videos. Only 3% (7 of 257) of websites included information in a language other than English. Patient narratives were found on only 15% (39 of 257) of websites. CONCLUSIONS: The field of Radiology is mentioned in text, images, or videos by less than half of LCS program websites. Most websites make only minimal use of multimedia content such as images, videos, and patient narratives. Few websites provide LCS information in languages other than English, potentially limiting accessibility to diverse populations.

Comparison of Chest CT Findings of COVID-19, Influenza, and Organizing Pneumonia: A Multireader Study.

BACKGROUND. Previous studies compared CT findings of COVID-19 pneumonia with those of other infections; however, to our knowledge, no studies to date have included noninfectious organizing pneumonia (OP) for comparison. OBJECTIVE. The objectives of this study were to compare chest CT features of COVID-19, influenza, and OP using a multireader design and to assess the performance of radiologists in distinguishing between these conditions. METHODS. This retrospective study included 150 chest CT examinations in 150 patients (mean [± SD] age, 58 ± 16 years) with a diagnosis of COVID-19, influenza, or non-infectious OP (50 randomly selected abnormal CT examinations per diagnosis). Six thoracic radiologists independently assessed CT examinations for 14 individual CT findings and for Radiological Society of North America (RSNA) COVID-19 category and recorded a favored diagnosis. The CT characteristics of the three diagnoses were compared using random-effects models; the diagnostic performance of the readers was assessed. RESULTS. COVID-19 pneumonia was significantly different (p < .05) from influenza pneumonia for seven of 14 chest CT findings, although it was different (p < .05) from OP for four of 14 findings (central or diffuse distribution was seen in 10% and 7% of COVID-19 cases, respectively, vs 20% and 21% of OP cases, respectively; unilateral distribution was seen in 1% of COVID-19 cases vs 7% of OP cases; non-tree-in-bud nodules was seen in 32% of COVID-19 cases vs 53% of OP cases; tree-in-bud nodules were seen in 6% of COVID-19 cases vs 14% of OP cases). A total of 70% of cases of COVID-19, 33% of influenza cases, and 47% of OP cases had typical findings according to RSNA COVID-19 category assessment (p < .001). The mean percentage of correct favored diagnoses compared with actual diagnoses was 44% for COVID-19, 29% for influenza, and 39% for OP. The mean diagnostic accuracy of favored diagnoses was 70% for COVID-19 pneumonia and 68% for both influenza and OP. CONCLUSION. CT findings of COVID-19 substantially overlap with those of influenza and, to a greater extent, those of OP. The diagnostic accuracy of the radiologists was low in a study sample that contained equal proportions of these three types of pneumonia. CLINICAL IMPACT. Recognized challenges in diagnosing COVID-19 by CT are furthered by the strong overlap observed between the appearances of COVID-19 and OP on CT. This challenge may be particularly evident in clinical settings in which there are substantial proportions of patients with potential causes of OP such as ongoing cancer therapy or autoimmune conditions.

Radiology Implementation Considerations for Artificial Intelligence (AI) Applied to COVID-19, From the AJR Special Series on AI Applications.

Hundreds of imaging-based artificial intelligence (AI) models have been developed in response to the COVID-19 pandemic. AI systems that incorporate imaging have shown promise in primary detection, severity grading, and prognostication of outcomes in COVID-19, and have enabled integration of imaging with a broad range of additional clinical and epidemiologic data. However, systematic reviews of AI models applied to COVID-19 medical imaging have highlighted problems in the field, including methodologic issues and problems in real-world deployment. Clinical use of such models should be informed by both the promise and potential pitfalls of implementation. How does a practicing radiologist make sense of this complex topic, and what factors should be considered in the implementation of AI tools for imaging of COVID-19? This critical review aims to help the radiologist understand the nuances that impact the clinical deployment of AI for imaging of COVID-19. We review imaging use cases for AI models in COVID-19 (e.g., diagnosis, severity assessment, and prognostication) and explore considerations for AI model development and testing, deployment infrastructure, clinical user interfaces, quality control, and institutional review board and regulatory approvals, with a practical focus on what a radiologist should consider when implementing an AI tool for COVID-19.

Pulmonary Vascular Manifestations of COVID-19 Pneumonia.

PURPOSE: To investigate pulmonary vascular abnormalities at CT pulmonary angiography (CT-PE) in patients with coronavirus disease 2019 (COVID-19) pneumonia. MATERIALS AND METHODS: In this retrospective study, 48 patients with reverse-transcription polymerase chain reaction-confirmed COVID-19 infection who had undergone CT-PE between March 23 and April 6, 2020, in a large urban health care system were included. Patient demographics and clinical data were collected through the electronic medical record system. Twenty-five patients underwent dual-energy CT (DECT) as part of the standard CT-PE protocol at a subset of the hospitals. Two thoracic radiologists independently assessed all studies. Disagreement in assessment was resolved by consensus discussion with a third thoracic radiologist. RESULTS: Of the 48 patients, 45 patients required admission, with 18 admitted to the intensive care unit, and 13 requiring intubation. Seven patients (15%) were found to have pulmonary emboli. Dilated vessels were seen in 41 cases (85%), with 38 (78%) and 27 (55%) cases demonstrating vessel enlargement within and outside of lung opacities, respectively. Dilated distal vessels extending to the pleura and fissures were seen in 40 cases (82%) and 30 cases (61%), respectively. At DECT, mosaic perfusion pattern was observed in 24 cases (96%), regional hyperemia overlapping with areas of pulmonary opacities or immediately surrounding the opacities were seen in 13 cases (52%), opacities associated with corresponding oligemia were seen in 24 cases (96%), and hyperemic halo was seen in 9 cases (36%). CONCLUSION: Pulmonary vascular abnormalities such as vessel enlargement and regional mosaic perfusion patterns are common in COVID-19 pneumonia. Perfusion abnormalities are also frequently observed at DECT in COVID-19 pneumonia and may suggest an underlying vascular process.Supplemental material is available for this article.© RSNA, 2020.

Right Ventricular Strain Is Common in Intubated COVID-19 Patients and Does Not Reflect Severity of Respiratory Illness.

BACKGROUND: Right ventricular (RV) dysfunction is common and associated with worse outcomes in patients with coronavirus disease 2019 (COVID-19). In non-COVID-19 acute respiratory distress syndrome, RV dysfunction develops due to pulmonary hypoxic vasoconstriction, inflammation, and alveolar overdistension or atelectasis. Although similar pathogenic mechanisms may induce RV dysfunction in COVID-19, other COVID-19-specific pathology, such as pulmonary endothelialitis, thrombosis, or myocarditis, may also affect RV function. We quantified RV dysfunction by echocardiographic strain analysis and investigated its correlation with disease severity, ventilatory parameters, biomarkers, and imaging findings in critically ill COVID-19 patients. METHODS: We determined RV free wall longitudinal strain (FWLS) in 32 patients receiving mechanical ventilation for COVID-19-associated respiratory failure. Demographics, comorbid conditions, ventilatory parameters, medications, and laboratory findings were extracted from the medical record. Chest imaging was assessed to determine the severity of lung disease and the presence of pulmonary embolism. RESULTS: Abnormal FWLS was present in 66% of mechanically ventilated COVID-19 patients and was associated with higher lung compliance (39.6 vs 29.4 mL/cmH2O, P = 0.016), lower airway plateau pressures (21 vs 24 cmH2O, P = 0.043), lower tidal volume ventilation (5.74 vs 6.17 cc/kg, P = 0.031), and reduced left ventricular function. FWLS correlated negatively with age (r = -0.414, P = 0.018) and with serum troponin (r = 0.402, P = 0.034). Patients with abnormal RV strain did not exhibit decreased oxygenation or increased disease severity based on inflammatory markers, vasopressor requirements, or chest imaging findings. CONCLUSIONS: RV dysfunction is common among critically ill COVID-19 patients and is not related to abnormal lung mechanics or ventilatory pressures. Instead, patients with abnormal FWLS had more favorable lung compliance. RV dysfunction may be secondary to diffuse intravascular micro- and macro-thrombosis or direct myocardial damage. TRIAL REGISTRATION: National Institutes of Health #NCT04306393. Registered 10 March 2020, https://clinicaltrials.gov/ct2/show/NCT04306393.